Cognitive cooperative assistance in robot-assisted surgery holds the potential to increase quality of care in minimally invasive interventions. Automation of surgical tasks promises to reduce the mental exertion and fatigue of surgeons. In this work, multi-agent reinforcement learning is demonstrated to be robust to the distribution shift introduced by pairing a learned policy with a human team member. Multi-agent policies are trained directly from images in simulation to control multiple instruments in a sub task of the minimally invasive removal of the gallbladder. These agents are evaluated individually and in cooperation with humans to demonstrate their suitability as autonomous assistants. Compared to human teams, the hybrid teams with artificial agents perform better considering completion time (44.4% to 71.2% shorter) as well as number of collisions (44.7% to 98.0% fewer). Path lengths, however, increase under control of an artificial agent (11.4% to 33.5% longer). A multi-agent formulation of the learning problem was favored over a single-agent formulation on this surgical sub task, due to the sequential learning of the two instruments. This approach may be extended to other tasks that are difficult to formulate within the standard reinforcement learning framework. Multi-agent reinforcement learning may shift the paradigm of cognitive robotic surgery towards seamless cooperation between surgeons and assistive technologies.

As robots replace humans in dangerous situations such as search and rescue missions, they need to be able to quickly assess and make decisions--to react and adapt like a human being would. Researchers at the University of Illinois at Urbana-Champaign used a model based on the game Capture the Flag to develop a new take on deep reinforcement learning that helps robots evaluate their next move. The team of researchers chose Capture the Flag because it's played with two teams, each with multiple teammates, where the opposing team is also making decisions. "Robots can learn how to react in an environment like a competitive game by using a kind of trial and error process, called reinforcement learning. They learn what actions to take in a given situation by playing the game," said Huy Tran, a researcher in the Department of Aerospace Engineering at UIUC. "The challenge is to figure out how to create agents that can also adapt to unexpected situations."